Conversation
boring-cyborg
bot
added
area:providers
provider:amazon
| yield from events | ||
|
|
||
| if 'nextForwardToken' in response: | ||
| if next_token != response['nextForwardToken']: |
There was a problem hiding this comment.
Should we use response.get('nextForwardToken', None) to avoid any chance of a KeyError here, or do we know for sure that will always be included in the response?
There was a problem hiding this comment.
Probably OK to just fail if the response format changes. An exception raised here (assuming the hook is used in a DAG) would eventually be caught by the test runner and logged to the task logs, which would clearly signal the problem (the API has an unexpected change in format). If we silently fall back to None, it would be more difficult to debug later if the API changed format unexpected (missing nextForwardToken), or actually returned "nextForwardToken": null (which is expected and signals a valid scenario, indicating we or the user may have logical bugs in the workflow).
There was a problem hiding this comment.
uranusjr covered my concern, but I'm also not a committer so my approval doesn't carry much (any) weight.
|
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
github-actions
bot
added
the
okay to merge
4 participants
Cloudwatch
get_log_eventscan return empty results even though there are more log events available in the stream. In my recent interaction with the AWS team, it was pointed out that the correct way to check for end of stream is that if the value ofnextForwardTokenis same in subsequent callshttps://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_GetLogEvents.html